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For sixty years educators have been concerned vith 
assessing staff "effectiveness." Perhaps we are still concerned with 
this problem because we have been chasing a will'-of --the- wisp. In a 
profession such as education, we should attenpt to aeasure staff 
coapetence rather than staf^ ef fecti^reiiess. There are three basic 
ways of assessing teaching: by using presage criteria (teacher 
characteristics) , by using product criteria (pupil gain) , or by using 
process criteria (teaching activitiess) . Teacher characteristics can 
be identified (if not aeasured) , but they do not differentiate 
teaching acts, student gain can be measured, but we can*t measure how 
auch is a result of the teacher's efforts and how auch is a result of 
variables the teacher can't control. Teaching activities can be 
observed and recorded as data, and, by using a criterion-referenced 
assessment instruaent (such as the Instrument for the Observation of 
Teaching Activities assessment program) , these data can be related to 
a set of educational expectations. Assessment will not improve 
teaching by itself; what is done to improve the conditions revealed 
by the assessment is of major importance. (Author/JG) 
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The topic today concerns assessing staff "effectiveness." 



GO 

^ Sixty years ago (in 1915) we were concerned about this very topic. Hazel 



Davis in the N.E.A. Journal ff February, 1965, quoted teachers as sayinn 
UJ "that teachers have the right to teach unhampered by any demeaning, 

artificial, arbitrary, perfunctory, and iuperficial rating.? 

Are there no new problems under the sun? Or have we net come 
very tar in solving tne assessment one? I suspect the ^atter— wt 
certainly have many new problems in secondary education; drug abuso, 
sexual permissiveness, student militancy, to name but three. 

Perhaps the reason we still have the same old "evaluation 
syndrome" is that we have been chasing a "will-of-the-wisp." 

I submit that in a profession you do not measure "effectiveness;' 
at least not if improvement is the objectivel What indicates an effective 
doctor? One who ne*'er loses a patient? Nonsense! One who loses only 
5%? Or one who correctly diagnosises all illnesses? Or is he one who 
loses 95^ of his patients because he deals only with the most difficult 
S cases? 

^ Is an "effective" lawyer o.ie who keeps all of his clients out 

§ of jail? 
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Is an "effective" minister one who keeps all of his parishioners 
out of the divorce courts and manages to arrange for every soul to enter 
into heaven? 

In spite of full divorce courts, full jails, and full grave 
yards, there must be some "competent" lawyers, ministers and doctors 
around. Oops— note, I changed a word— I spoke of "competence" not 
"effectiveness." 

With no desire to play a semantic word game, let's look at the 
difference. A doctor performs an operation and the patient dies— does 
this mean that the doctor was not competent? Perhaps, but more likely 
It means that he was not effective. 

What gives? 

Webster defines "effectiveness" as "producing a decided, 
decisive, a desired effect. Competence" is defined as "having (the) 
requisite ability or qualities." 

I maintain that we have been looking in the wrong direction in 
the teacher assessment area. 

There are three basic ways, as we see it, of assessing "teach- 
ing." (Show transparencies) 

1. Presage criteria - Teacher characteristics 

2. Product criteria - Pupil gain (perhaps the ultimate 

criteria) 

3. Process criteria - Teaching activities. 

Dr. Ryans of the University of Hawaii, who has spent a life- 
time dealing with teacher characteristics, told my associate Dr. 
Carpenter recently that we are still in the "stone age' in attempting 
to assess teaching through "characteristics." It is not that they 
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cannot be Identified (I am not so sure that they can be measured) but 
they do not different- 'te the teaching acts. 

In the paper, "Characteristics of Good Teachers and Implications 
for Teacher Education," Don Hamachek said that students wanted teachers 
^ who were "good people," "flexible," "likeable," "total," etc. So do I, 
and I also want these characteristics in my wife, the milkman, and the 
local dog catcher. 

Enough of characteristics— even though most of the 3,200 
evaluation instruments (the latest count with which I am familiar) are 
composed largely of such items— e.g. "possesses growth potential." 

Well, let's go to the other end of the continuum. There is the 
"student gain" approach. Certainly this is the ultimate criterion. 
The problem here is not so much that it cannot be measured— tr> limited 
degrees it can (we are not always certain what we are measuring, even 
in the cognitive area); what we can't re illy do with any degree of 
certainty is to say, "Johnny— nut?, I hate the name Johnny these days 

■m 

since you know v/hat— let's say, "Bill," didn't learn because "Miss Susan" 
is a poor teacher. Vie can't even be certain that Bill did learn (note 
the "Coleman Report"}^ because of her; yet soniahow we don't worry about 
that aspect the problem. 

We can't be certain that he didn't "jearn because of "Miss Susan" 
because "Miss Susan" can't control the variables. She does not control 
his health, the lateness of the hour that he goes to bed, the fact that 
his father beats him everytime he gets drunk, and on, and on, and on... 

And we have not even spoken of Bill's innate ability to learn. 
What is the learning potential of a twelve year old, blond, 5-6th grader, 
Swedish-Italian male, with an I.Q. of 100 and reading skills "on grade" 



ERIC 



4 



coming from a home with little or no reading material? Also, what about 
his previous educational experiences? You would have to Isolate him 
aod teach him in Skinner's so called "cage" to even come close to 
determining the learning potential. 

Neither the doctor, the minister, the lawyer, nor the teacher 
can be held accountable for their "effectiveness." 

Lessinger 1s quoted in the January 21st, 1974, issue of 

Education U.S.A. that to hold one party accountable for the acts of 

another was unreal. Note his further statement: (Show transparency) 

(1) The only thing a professional can be held 
accountable for is using souvid practice. . . (2) personal 
accountability to the students, and (3) accountability 
to the profession... 

Yes, we can be accountable for the things we do— and I don't 
mind so being held. In my work across the nation (presently with 
around 1,000 teachers a year for 36-40 hours) I find very few who 
object to being held to what I have come to call, "process account- 
ability." 

The things v/e do are observable or at least ore verifiable. Vie 
who spent tine in New York state know that some surgeon left a "sponge" 
inside Congressman Carlton King at Bethesda, Maryland, a few years 
ago. That v/as both observable and verifiable, and while the operation 
was effective (even the re-opening) that act wasn't in my opinion very 
competent. 

I am sure that somewhere there is written instructions on how 
to perform an operation. I am certain that a phase somewnat akin to 
"be sure to remove all 'sponges'" is written down. 

Hence, the next point 1 Since we can observe and verify what 
is done in the classroom, the behaviors of the teacher end students, 



these can be recorded as data. But how to relate this data to 
expectations? It seems to me two approaches are possible. The norm 
reference approach or the criteria reference approach. 

Using a norm referenced procedure hundreds and hundreds and 
hundreds of observations are completed and analyzed; then all are added 
up and divided by the total number of observations giving a "norm of 
expectations." We could then say that most teachers do this so all 
teachers should do so as well. But the norm is an average and I am 
not interested in averages I 

Well, the other approach then is the criterion reference 
approach. Define what is to be done specifically; develop instrumen- 
tation to determine if the behavioral tasks so defined are being accom- 
plished; train the people to use the instrumentation; cause them to 
become familiar with the definition, and allow for some local adaption 
of the instruiiientation. 

In a profession the only v^ay you can detenu* ne what the tasks 
are that need to bo done is tsk the practitioner and the recipiants of 
the sevvice. If teacners (I include principals as teachers) and 
parents can't tell you what needs to be done to produce "learning" then 
no one can I 

Now there ar-:: a number of behaviorc.1 approaches to the assess- 
ment of instruction. "Mirrors for Behavior" lists over ninety. 

In iny work, limited as it is, I know of only one all inclusive, 
specific, written and comprehensive definition of teaching. It was 
known originally as The Measure of a Good Teacher and was developed in 
1952 by the California Council on Teacher Education and was adopted 
by the California Teacher's Association Teacher Education Committee 



in 1955. Now known as The Role of the Teacher in Society-Six Areas 
of Teacher Competence , this all inclusive definition of teaching 
competence has been recognized by the National Commission on Teacher 
Education and Professional Standards of the National E<^ucation 
Association (1964). 

The Role of the Teacher in Society breaks the teaching acts 
down into 100 observable or verifiable behavioral acts. These 100 
teaching or professional acts (for all are not necessarily classroom 
acts) are further subdivided into six major areas of teacher competence: 

1. Director of Learning 

2. Advisor and Counselor 

3. Mediator of the Culture 
A. Link with the Community 
3. Member of the Staff 

6, Member of the Teaching Profeiisiou 

The old Thcrndyke principle still applies. Once defined some- 
thing can be treasured. The IOTA (The Instruinent for the Observation 
of Teaching Activities) an assassrn^nt instruir.cnt, is designed to reflect 
through twenty-seven "scales" (each ccmposed of five behavioral items) 
the factors of the definition just discussed. 

The program with which I am associated is founded upon the 
definition, The Role of the Teacher In Society . People are trained 
to use the instrumentation and we continue to work with districts as 
they develop a planned program of professional improvement.^...''''''''^ ^..^^^ 
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But let's also remember that the assessment of teaching will not 
improve teaching. What is done to improve program conditions as 
revealed by the assessment is far more importanti! For this, four 
elements are necessary. 

1. First, and perhaps paramount, a desire and commitment on 

the part of all for improvement must exist. The period of "lip service" 
to improvement has slipped away. 

2. Time— staff improvement takes time— not just administrative 
time— but total staff time— Goodlad has said it for us. 

Show Goodlad's slide! 
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Teachers need time to observe what they are 
doing... and they need training in how to observe what 
they are doing. They need access to resources, to know 
the possible programs and assistances that are avail- 
able... They need support and encouragement. 

3. Money-time is money— the re -development of staff, the train- 
ing of a total icarr, etc. all require, wnile not excessive, th? 
expenditures of funds beyond t.^at usually allott^^d to "R. and D." 

Kew texts are iir.portant, new equipment also; but the postponenient of 
a total text adoption across the total school would provide more than 
sufficient funcs to accomplish the initial training of a school staff 
tov.'ards self-improver.x-nt through a criterion referenced procedure. 

4. Willingness to re-ord2r traditional operational procedures-- 
a peer assessment, which is the only one that can accomplish the task, 
program requires that things be done differently in order to provide 
the manpower to "field a total improvement program." There are no easy 
solutions to staff improvement. 



ERIC 



9 



"TEACHERS NEED TIME TO OBSERVE WHAT THEY 
ARE DOING. ..AND THEY NEED TRAINING IN 
HOW TO OBSERVE WHAT THEY ARE D0IN3. 
THEY NEED ACCESS TO RESOURCES, TO KNOW 
THE POSSIBLE PROGRAMS AND ASSISTANCES 
THAT ARE AVA I LABLE. . .THEY NEED SUPPORT 
AND ENCOURAGEMENT. " 




Goodlad 

Report on Education 
ajiu^ry 30, 1974 
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So it woula appear that we, in my upinion, can assess competence 
but not effectiveness. Can such a program be called accountable? 
Let's check against the AASA's Handbook on accountability (1973). 

The Imperatives for accountability (in my context) "process 
accountability" are: (Show AASA transparency) 

1. It must have knowledgeable designers— many programs do. 

2. It must lead to improved education— one, at least shows the 
development of considerable change, towards the definition, in the ways 
teachers perform the teaching act and in their acceptance of students. 
The definition says this is "competent teaching," and I believe it. 

3. It must recognize and accommodate diverse forms of partici- 
pation—many programs do— The criterion referenced one to wnich I 

. referred, operates very successfully K-14. 

4. The program needs trained personnel before and during 
implementation. This is a key to all improve«Tient programs. 

5. The program must fulfill .:.e conditions cf the accountability 
concept. Does it have (joals, pi ins, 2 rathod of developing a procedura 
and process and organisation to carry cjt tha cLjectives? Are all of 
these verifi-jble? We believe at least one program does. 

6. The program must be judged politically obtainable. Does the 
where 'Wi thai 1 and ability exist to carry it off? If a sponsoring grouo 
is not willing to spend the time, effort, and funds over the long term- 
forget iti The professional development of teachers is not a short 
range project . 
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I firmly believe that the profession holds within its hands, 
right now, the ability and knowledge needed to improve instruction 
through a procedure which I call "process accountability." I am 
firmly convinced that we can do it if we are devoted to this task and 
do not cloud the issue with threats of punishment or reward. 



